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ABSTRACT 

This report describes procedures for effective 
development and implementation of workplace literacy programs based 
on the experience of a partnership among Alabama Southern Community 
College (Monroeville, Alabama), Boise Cascade (Jackson, Alabama), and 
Vanity Fair (Monroeville, Alabama). The introductory summary 
describes the voluntary, after-hours project to improve the basic 
skills of 539 workers at the two companies in job-related reading and 
math. Subsequent sections contain the following information: a 
summary of future work force needs, definition of workplace literacy, 
the importance of a linkage to total quality management, partner 
selection, program structure, program promotion, and external 
factors. Also included are 15 references, a promotional flier, 
personal profile form, and an external evaluation report. (NLA) 
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PURPOSE STATEMENT 



The purpose of this report is to provide procedural guidance for the effective 
development and implementation of a national workplace literacy project. A 
tremendous quantity of information on this subject is readily available. There are 
thousands of varied sources which supply those seeking direction with the 
fundamentals: literacy testing, needs analysis, task analysis and the like. There 
appears to be a void in the area of recommendations which address effectiveness. 
This report will, therefore, take a real world approach to the management of a literacy 
program. 

General information will be given regarding the origin of workplace literacy, its 
definition and its tie-in to total quality. Specific knowledge regarding implementation 
will also be shared. The areas addressed are partner selection, program structure, 
program promotion and external factors. 

Because of the dynamics involved no two projects are identical in management 
needs. It is hoped that the reader will gain insight into the often unanticipated 
barriers which can jeopardize a project's effectiveness. 



CASE SUMMARY 



The project at hand was a demonstration National Workplace Literacy Project 
funded for the 1991/92 cycle. It was a partnership among Alabama Southern 
Community College (formerly Patrick Henry College), Monroeville, Alabama, Boise 
Cascade, Jackson, Alabama and Vanity Fair, Monroeville, Alabama. The project was 
the first workplace literacy grant received by the college. 

The target participant number was 500. The total number of participants 
tested and involved in the project was 539. After surveying many users of 
measurement instruments, the TABE test series was selected. The levels and 
duration of participation varied greatly from operation to operation and of course, 
from partner to partner. 

A determination was made early into the project that there was a negative 
stigma attached to the word "literacy." The term workplace "education" was 
substituted. Even so, perception of the project as a remedial program was hard to 
overcome. 

At Vanity Fair, the project was expanded to include all operations in 
Monroeville, the dye house, administration, distribution center, cutting and sewing 
plants. Additionally, the project was geographically expanded to include Vanity Fair 
sewing plants in Jackson, Alabama and Demopolis, Alabama. 

No release time was given for job-related skills enhancement at Vanity Fair, so 
the project was totally voluntary and after hours. After initial meetings with 
operational personnel managers, then with supervisors and finally with participants 



on a one to one basis, shifts of instruction were set up in each operation as 
participation merited. 

Attempts were made to further the familiarization of project personnel with 
operational vocabulary and develop a company profile. Organizational charts were 
requested, but denied. This hampered the development of job-related skills training. 
Some testing and workplace materials were shared. While certain segments of the 
Vanity Fair operation did well on the TABE, distinct math and reading deficiencies 
were noted. 

Post-testing was difficult. However, on those participants post-tested from the 
first cycle, a 10% overall gain in skills was measured. 

Personal profiles were developed for each participant, as well as an 
individualized course of instruction. A copy of the profile form is attached as an 
exhibit to this report. 

Retention rates were not high even though the program was confidential and 
portable in nature. Scattered success stories, such as promotion and GED 
obtainment could be found through out the Vanity Fair operations. 

The majority of instructors employed in the Vanity Fair component of the 
project were already involved in some adult education activity in the community. 
They were personable and casual in attire, a request from partner management. 

The voluntary component of the Boise Cascade participation evolved through 
the same* series of meetings. A full time instructor with a math emphasis was 
housed in the training center. An applicant was selected after all finalists toured and 
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reacted to the mill environment. Additionally, a skills bank was installed so that 
individualized instruction, both computerized and personal, could occur. 

The individuals at Boise, for the most part, scored at the high end of the TABE. 
There were some exceptions in the area of mathematics. 

Job-related advanced math skills were taught to electrician apprentices at 
Boise. Ail participants, except one, advanced to the required level of knowledge. 

Life skills, or personal development skills, were furnished at no cost to the 
project to both partners by the state extension service. Examples of topics addressed 
were stress management, gun safety, financial budgeting and assertiveness.' The 
financial planning was especially helpful at Vanity Fair given the occurrence of 
garnishments. 

Major differences in the partners existed. f)n» insisted on a totally voluntary 
project and was very decentralized with many layers of management. Because these 
managers were autonomous establishing a solid niche for the program was difficult 
with this partner. However, release-time participation, when closely associated with 
a centralized training program during economic downturns, was also detrimental to 
the project. This was evident by the reactions experienced at the second partner's 
operation. 

It would be easy to cite the level of company involvement as a critical success 
factor to this project. However, external factors, which will be discussed later in this 
report, can make this an untrue correlation. 

The real success of this project can be found in the fact that it was 
implemented and as a result both employer and employee awareness was heightened. 



The awareness level moved beyond the two partners and into the business and 
industry community in the service area as a whole. Both companies are subsequently 
involved in job-specific release time training. 
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GENERAL INFORMATION 
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WORKFORCE 2000 



Corporate America was once seen as invincible. "In the first half of this 
century, rapid expansion, rising productivity, and seemingly unlimited market 
opportunities were commonplace for American business." (Handshear, 1990) This 
is no longer so. 

Much has been published and said about America's decline in industrial 
competitiveness. The underlying reasons have been thoroughly examined and 
debated. 

The most alarming and heeded of these publications was a 1 987 report entitled 
"Workforce 2000." The project was jointly undertaken by the Employment and 
Training Division of the U. S. Department of Labor and the Hudson Institute. Specific 
issues cited by this report were a slump in workforce growth to 1% per year; an 
older, more female, and more disadvantaged workforce; an expanded role for family 
matters and a widening skills gap (Klein, 1990). 

This report became the basis for National Workplace Literacy programs. 
Subsequent figures estimate tie annual dollar loss of productivity from a poorly 
educated citizenry at $240 billion (Kindel, 1988). 

The economic base of the United States has changed from an industrial focus 
to that of service oriented information focus requiring increased skill levels. "Not only 
are the number of functionally illiterate rising, they are becoming more noticeable, 
particularly as manufacturing and 'hands-on' jobs give way to an information driven 
society." (Pilenzo, 1990) 
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It is widely held that the United States is not appropriately addressing the skills 
gap. Nothdurftin 1990 stated: 

In short, while the United States continues to worry about the competitiveness 
of companies, many European nations are investing in the competitiveness of 
individuals, believing that when individuals are competitive, companies and 
nations will be competitive as well. 

A more recent report, "Workforce 2000; Competing in a Seller's Market: Is 
Corporate America Prepared?" implies that the much anticipated changes have 
already arrived. This report was a joint project of die Hudson Institute and the 
Towers Perrin consulting firm. The report goes one step further in stating that 
corporate America is unresponsive. "Two demographic trends they [corporate 
America] don't seem worried about: the aging workforce (companies are still 
encouraging early retirement); and the job applicant skill levels, even though the 
'skills gap' is projected to widen dramatically." (Beilinson, 1990) 

The findings of this report were based upon surveys of 645 human resource 
managers. One of the suggested responses in this report was to "[t]ie basic skill 
training as closely as possible to concrete work tasks" (Beilinson). This conclusion 
reiterated a continuing need for workplace literacy. 

It is strongly suggested that a copy of each report be obtained and read before 
the implementation process begins. Insight into the future needs of industry partners 
can be gained. Doing so also provides a window for future business and industry 
training opportunities. 
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WORKPLACE LITERACY DEFINITION 



The skills gap exists and it is widening. That is a substantiated fact. What 
constitutes an acceptable level of literacy in the workplace is not as well defined. 

It has been said that workplace literacy projects should not supersede and 
duplicate existing adult basic education programs. How does one move up to a 
higher skills level if deficiencies exist in basic skills? 

It has been said that workplace literacy projects should not supersede and 
duplicate existing vocational and equipment simulated training. This is certainly 
understandable. 

Does workplace illiteracy exist only in low level positions? Are there not skill 
deficiencies relevant to every level in an organization? For example, increasing 
responsibilities demand increasing levels of communication skills, both oral and 
written. 

Workplace literacy's purpose as cited by Sticht in 1991, is "...to improve the 
productivity of the workforce through the improvement of literacy skills in the 
workplace by: 

(a) Providing adult literacy and other basic skills services and activities; 

(b) Providing adult secondary education services and activities that may lead 
to the completion of a high school diploma or its equivalent; 

(c) Meeting the literacy needs of adults with limited English proficiency; 

(d) Upgrading or updating basic skills of adult workers in accordance with 
changes in workplace requirements, technology, products or processes; 

(e) Improving the competency of adult workers in speaking, listening, 
reasoning, and problem solving; or 
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(f) Providing educational counseling, transportation, and child care services for 
adult workers during nonwcrking hours while the workers participate in the 
project (Federal Register, August 18, 1989, vol. 54, no. 159, p.34418)." 

Workplace literacy is all of the above and more. It is also the securing of 

personal development materials, such as those available through the local extension 

service. 

The above skills are often referred to as life skills. A 1990 article regarding 
literacy training in hospitals sums up the relationship between life and lite; ? ay skills 
as: 

If employees cannot figure out how to budget, pay for child care, balance their 
checkbook, or sign up for medical insurance, their productivity will be 
hampered. In response to this reality, some [hospitals] are offering iife skills 
as well as basic literacy training. We need to help people along. This [life 
skills] has become a necessary part of the workplace. (Hospitals, 1990) 

Direct inquiry regarding a hard and fast definition for workplace literacy 
remained unanswered at the National Dl "tors' meeting. Emphasis was placed 
again and again on tying all workplace IH^racy projects into the partner's efforts 
toward increased productivity. 

The connection between total quality, productivity and workplace literacy is 
clear. The three are an integral and inseparable trio in building a competitive 
workforce which was the original premise for the funding of workplace literacy as 
defined in "Workforce 2000." 

Workplace literacy training can encompass any skill training, excluding 
instrumentation, for which the lack of hampers the productivity of the partner 
organization. These deficiencies exist in every level of an organization. 
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TOTAL QUALITY LINKAGE 



A total quality or continuous improvement management approach can never 
be completely effective if the literacy or job related skills are not first addressed. 
"The quality improvement process presupposes that employees have these basic 
attributes. When they do not, it is necessary to offer them the opportunity to learn." 
(Roach, 1990) 

Frequently, the lack of job related basic skills is only evident when an 
organization undertakes a quality program. Deficiencies are exposed because of the 
empowerment and accountability components of these programs. 

One of the fourteen principles of Dr. W. Edwards Deming's total quality 
approach specifically states the need for vigorous training at all levels. To increase 
productivity, an employee must have all skills necessary to do the job. 

A 1992 article cites the ineffectiveness of many quality programs "...when 
they [organizations] conducted awareness training about quality, but failed to follow 
up with skills training... employees were frustrated.... The improvement effort had lost 
credibility. It is difficult to reverse such feelings." (Ferketish and Hayden, 1992) 

To take the premise of "Workforce 2000" one step further, management is 
concerned about productivity because of its effect on the organization's bottom line 
or profit margin. The need for positive financial results is the strong connection 
between 'any continuous improvement program and workplace literacy project. 
Executives are beginning to realize if they do not take the initiative in preparing their 
workforces for the future, their profits will ultimately suffer. Many executives are 



ERIC 



14 



beginning to tailor programs to improve and expand their companies competitive 
advantage (Greenberger, 1991). 

Today's skill enhancement and measurement is not broad in nature. " ...[A] lot 
of our training speaks to the competitive aspect of our industry and no longer 
involves a general menu of training activities. High on the list are customer relations, 
efficient management -and perhaps most importantly - the establishment of a 
'team' mentality." (Greenberger) 

The trend toward just in time, very specific skills enhancement creates a 

perfect niche for workplace literacy. If the predictions of "Workforce 2000" and 

subsequent reports hold true: 

...-companies will have to invest in remedial education by directly assisting 
schools and through on-site programs for new hires. They also will have to 
invest more in job-specific training because of more advanced technology. 
Career paths at all levels will be very important (Elllg, 1990) 

The need for greater profit margins creates the need for increased productivity 
and higher customer satisfaction which create the need for continuous improvement 
and total quality. None of these can ever be fully achieved when essential job-related 
skills are missing. An employee who is lacking in self-esteem and self-confidence due 
to skill deficiencies is very unlikely to be the one to suggest process improvements. 

This relationship is described by Nothdurft. "Product quality enhances 
competitiveness, workforce education and training enhances product quality. Pay 
now, reap the benefits later." One cannot exist independent of the others. 
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PARTNER SELECTION 



The selection of appropriate partners is critical to the success of a workplace 
literacy project. The time lag between submission of the proposal and funding 
notification can create difficulties in maintaining enthusiasm levels. 

A partner who has an ongoing training program will be more likely to remain 
excited about the project. This is important because projects of less than 18 months 
do not have an appropriate amount of time to establish the needed awareness. More 
importantly, they do not have time to become institutionalized and accepted. The 
level of commitment of a partner to training can be critical here. 

The preceding paragraph should not steer institutions away from small 
companies. Although in house training is not as prevalent in small businesses, they 
can be an excellent partners. In the future, "...[mjost new jobs will be created by 
entrepreneurs and small firms...." (Warren, 1990) If the current pattern continues, 
the majority of new hires will require both remedial and job-related training. 

The needs of small businesses for workplace literacy may be greater than their 
larger counterparts. They will really feel the squeeze of the predictions of "Workforce 
2000." Because they cannot compete with the higher salaries offered by big 
business they will be seriously impacted by shrinking tabor pools with lower skill 
levels (Klein, 1990). 

Another interesting factor about small businesses is that they invest in and 
care about their workers and the communities in which they operate (Jasinowski, 
1991). Because of high levels of community involvement the owners of small 
businesses are more receptive to innovative concepts, such as workplace literacy. 
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The demographics of an area should be considered when making a partner 
selection. Rural service areas may lack the critical mass necessary for an ongoing 
project. If this is the case, then the formation of a training consortium whether of 
a single industry or not can be effective. This provides an excellent opportunity for 
information sharing and a chance to make an impact upon the productivity of a 
region. 

Look for a partner who is familiar with the team approach and utilizes it on a 
daily basis. Ask for an implementation team. Getting the appropriate input and 
feedback is easier when working with a team. Dealing with only one person at a 
company or with many fragmented equivalents can be detrimental to the project. 

Finally, it goes without saying that an institution needs a partner who is willing 
to give at least some release time for project participation. Whether it is shared or 
totai release time does not matter. Participants want to see some organizational tie- 
in. Examples are pay-for-skills and other incentive programs. They want to know 
management is behind and fully endorsing the project. Release time is the best 
evidence around. 



PROGRAM STRUCTURE 



As previously stated, the characteristics of a workplace literacy program can 
and should vary from operation to operation, as well as from partner to partner. 
Flexibility is important. 

If a program is flexible, customer driven and future oriented, then a total quality 
or continuous improvement approach to structuring is being utilized. Do not be afraid 
to ask all levels of employees what types of skills training they need. They are, after 
all, the project's customers. 

A workplace literacy program is best structured when it adheres to total quality 
principles and in fact, is seen as a component of the partner's total quality program. 
It will then be perceived as a vehicle for increased productivity, increased bottom-line 
and increased financial well-being for the employees as well as the partner. 

Structure a project so that just in time training can be provided. Make sure 
that an actual company assignment is tied to a favorable outcome. Even better, 
make sure that the skills are immediately applicable to the workplace. 

The more forms, pamphlets, posters, that can be gathered from a partner, the 
better. Encourage instructors to make the skills taught as realistic as possible by 
using examples from the workplace. The instructors should always go through an 
orientation to the project, as well as a review of the appropriate company profiles. 
Nothing builds credibility faster than speaking the same language as a participant. 

If a project is totally committed to continuous improvement, there will be no 
hesitation about inquiring as to what is being done wrong, as well as to what is being 
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done right. Questions of this nature will make it easier to insist upon measurement 

of instructional effectiveness. A related pretest and posttest assures that a project 

is adequately meeting the job-related training needs of the partner. 

The Society of Human Resource Managers has identified certain characteristics 

that all successful workplace literacy programs share. Pilenzo lists them as: 

Basic skills training is packaged as part of a broad training agenda which 
encourages participation. 

Top level management, supervisors and workers are all involved in the planning 
process. 

Employees' personal; goals are solicited and included in program planning. 

Instructors are aware of the basic skills needed to perform the specific job 
tasks for which they are providing training. 

Program goals and standards for measuring progress are clearly specified; 
pretests and posttests simulate job situations and tasks. 

Learning materials are directly related to the goals. 

Feedback is frequent and progress is documented. 

Where possible, incentives such as the opportunity to learn new technology 
are provided to qualify employees for new job openings, or to meet personal 
goals. 

Training is scheduled wholly or partially on company time to encourage 
attendance. 



PROGRAM PROMOTION 

t 

The first and foremost rule to remember in the promotion of a workplace 
literacy project is to never underestimate the power of the supervisors and the 
company grapevine. Misinformation is extremely damaging to a project's credibility. 
Steps must be taken in the planning process to assure that all information 
disseminated is accurate and reflects the true spirit of the project. 

Overcoming the "literacy" stigma is difficult at best. In many business 
environments, employees place deep seeded trust in the opinions of their front line 
supervisors. Without the buy-in of this group, recruitment is almost impossible. If 
supervisors regard the project as a dummy, remedial program, so will the individuals 
who work for them. 

» 

Talking about the project is not enough. Visible evidence of commitment must 

be demonstrated by management. Pttenzo stresses the importance of commitment. 

It Is extremely important for the company not only to show commitment to an 
educational program, but to show respect for the enrolled employee. Very few 
employees will enroll in a program if they feel management resents the program 
offered or that their job wUI be in danger by revesting skin deficiencies. 

This fear can be especially sabotaging to the all voluntary after houra project. 

Provisions must be made to overcome this reluctance. 

The moat important step in promotional process is to maintain a consistent and 

believable demeanor. "Forget the fanfare and the banners...(t]o moat workers, 

kickoff speeches and pep rallies don't signal management commitment; they signal 

a new 'program of the month.'" (Hughes, 1991) A project cannot be aK things to all 

people in the company. It is better to have a committed few who achieve measurable 

results. 
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Never promise a benefit or result that cannot be delivered. If improvements 
in productivity or positive cost/benefit analysis trackable to the project are achieved, 
this information should be shared with everyone. Management will always appreciate 
a cost reduction. In times of economic downturns, workers at ail levels will be more 
focused upon cost reduction. 



EXTERNAL FACTORS 



Certain barriers to the success of a demonstration project, which is completely 
voluntary and after work hours, are obvious. Examples of such problems are money, 
time, child-care and transportation. 

External factors which can impair the effectiveness of a project are not always 
as apparent. These factors are external to the provider of the project services. They 
may be internal to the recipient. 

The effect of a recession upon a workplace literacy project can never be fully 
or accurately anticipated. There are two schools of thought. With layoff's and 
rumors of impending shutdowns, training is usually the first function sacrificed. If a 
workplace literacy program is closely associated with a centralized training 
department, then the project will suffer the same consequences. 

In the case of a voluntary, after hours, project, participation may increase due 
to the perceived need to be more competitive through higher skill levels. This is 
probably the single most unpredictable external factor faced. 

Industries which are cyclical In nature can present problems for a project. The 
external factor of extreme levels of overtime can affect participation. The flow of 
industrial cycles, either with overtime or layoffs, must be incorporated in the planning 
stages of the project. 

When a partner exists within an industry that is highly competitive and subject 
to rapid demand changes, it may turn from the very thing that could give it 
comparative advantage -higher skill levels. The tendency is to react to market 
swings through cost cutting rather than to proact through increased training. 
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Reorganization of a partner can also affect a project. The trend for flatter 
organizational structures has necessitated reorganization. As should be anticipated, 
managers concerned with keeping their positions will not be as committed to the 
promotion of workplace projects. 

It is important to be familiar with the external structure of each partner's 
industry. By doing so, preparation can be made for any trickle down effects. 
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CONCLUSION 



A workplace literacy program is always most effective when it is embraced as 
an opportunity rather than confronted as a threat. It is up to the provider to create 
this relationship. 

Evaluation of the program is an ongoing process with little room for expanded 
egos. The objectives of the project must be realistic and obtainable. Results 
enhance your credibility with both partners and participants. 



25 



REFERENCES 

Beilinson, J. (1990). Workforce 2000: Already Here? Personnel . £7.00), 3-4. 

Ellig, B. R. (1990). Controlling HR Costs. Personnel . £2(4). 20-24. 

Ferketish, B. J. & Hayden, J. W. (1992). HRD & Quality: The Chicken or the Egg? 
Training & Development. 4£(1), 39-42. 

Greenberger, L. S. (1991). Employee Training: Investing in Human Capital. 
Public Utilities Fortnightly. 128(5). 23-27. 

Handshear, N. (1990). Total Compensation: Trends in Employee Compensation and 
Benefit Programs. Management Quarterly . 3JL(2), 47-49. 

Staff. (1990, January 20). Literacy Crisis Threatens Hospital Workforce. 
Hospitals , pp. 50,52,53. 

Hughes, B. (1991). 25 Stepping Stones for Self-Directed Work Teams. Training. 
2&(12), 44-46. 

Jasinowski, J. (1931). The Magic of Small Manufacturers. Industry Week. 
240(16). 35. 

Klndel, S. (1983). Education's Crisis. Financial World . 157.(22), S21-S22. 

Klein, E. (1990). Tomorrow's Work Force. D & B Reports. 25(1), 30-35. 

Nothdurft, W. E. (1990). How to Produce Work-Ready Workers. Across The Board . 
21i9). 47-52. 

Pilenzo, R. C. (1990). Why Literacy in Everyone's Business. Modern Offflce 
Technology. 35(1 1 ), 82-84. 

Roach, J. B. (1990). Education and Training at Welch Allyn. ILR Report. 
25(1). 14-17. 

Sticht, T. G. (1991). Evaluating National Workplace Literacy Programs. 
El Cajon, California: Applied Behavioral & Cognitive Sciences, inc. 

Warren, A. S. (1990). Creating a Competitive Workforce. Economic D evelopment 
Review. fid), 10-12. 



EXHIBITS 



ERIC 



27 




ii II 



IS 
if 



ii 3 



t 
l 

i 

illii 

ilu 



S 



il 5 



» 

» 



o 

X 

CL 





UJ 

oc 

Ui 

u o 

5"- 

III 
5 




o 

>- 




PERSOHAL XOTOIQUTXOM 

1. My name is , 



2. i am a (male/ female) 

3> z m years old and I live with my 

(father, mother, husband or wife. 

family, alone) 

4. My favorite day of the week is 



5. The TV program I like the most is 



6. My favorite course in school is/was 



7. I like (ed) it because 



a. The course in school I dislike/disliked most is/ 



9. 1 dislike led) it because 



10. In my free time I really enjoy 



11. My job now is 



PERSONAL PROFILE FORM 
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12. Write 3 or more sentences about yourself. 
Spend 5 minutes 



13. If you would like to, please write a few sentences about the 
person you would most like to meet and why you would like to 
meet him or her. 



14. What are your personal goals for the future? 
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EXTERNAL EVALUATION REPORT 
BY 

DR. BOBBY DEES 
FOR 

ALABAMA SOUTHERN COMMUNITY COLLEGE 
WORKPLACE EDUCATION PROGRAM 
AUGUST 24, 1992 



NATIONAL WORKPLACE LITERACY EVALUATION 
EXTERNAL EVALUATOR 
BOBBY DEES 



1. Did this program demonstrate a strong relationship between skills and the 
literacy requirements of actual jobs or increased skill requirements of the 
changing workplace? • , , . j ol (W* >£ ^ m,> 

2. * Did'instruction take place in a readily accessible environment conducive to adult 

learning? L( i <t ! ^ r ^ H *jA~*. ""^ >~ 



3. Did curriculum materials used reflect the needs of the workplace? 

Were instructors qualified to provide literacy services to working adults? 



5. Was training provided^wjthin the framework of the project? 

6. Were support services provided such as educational counseling? ^ £ 5 / 

7. Were benefits of the Project defined to adult workers and their industries? L(45 . 
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8. Were individualized educational plans developed jointly by instructors and adult 
learners? Lf 4 s , ^ , 

9. Were effective measures available for participants to gauge whether personal 
goals could be obtained such as career advancement; continued employment; 

or increased productivity? / , , . * ^ . 

10. Were participants selected without regard to race, color, national origin, gender, 
age, or handicapping condition? 0^ 5 [ 

1 1 . Were the performance levels of the participants measured? (^^^ \ 

12. Did partners actively commit to accomplishing project goals? e^U j 

13. Were respective roles of each partnership clearly defined? ^ q_ <==~^ \ . 
\W\U\^ *^ O.^n^N^^^j ^MvyVLw^c^Jr V^e^r^-^^. 

14. Was data periodically measured and used by the project for program 
improvement? j%j\J 
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